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ABSTRACT 

While it is well-documented that appropriate training of teachers in the use of technological tools is an important 
variable in predicting the success of technology integration (Faseyitan & Hirschbuhl, 1994], a variable that is less often 
considered is that of the time spent or different levels of immersion in a technology integration training program 
(Stuhlmann & Taylor, 1999], This factor must be considered to be separate and distinct from the variable of overall 
experience as a teacher in the classroom, and/or teacher age. 

This paper is based on a selected portion of the results of a five-year evaluative study of the teachers implementing a 
technology integration program in a rural school district located in the Southwestern United States. The fifth year design 
included analysis of observation data byyear(s) of teacher participation in the program, as Novice, havingl to 2 years of 
experience ; Experienced with 4 to 5 years, The purpose for the focus on this aspect of the evaluation was to determine 
what differences, if any, existed between novice (to the program training] and experienced (in the program training] 
teachers in how and to what degree technology is integrated into classroom instruction. The findings reflect differences 
of varying degrees in certain areas of instructional strategies and technology integration that should be considered for 
further investigation. 

Keywords: Technology integration, teacher preparation programs, levels of program immersion. 


INTRODUCTION 

Few would argue that the advent of new technologies has 
had a substantial impact on the nation's society, culture, 
and workplaces, However despite the prevalence of 
computer placement in the United States' K-12 
classrooms, the measured impact of technology on U.S. 
education has been relatively minimal (Norris, etal, 2003; 
Cuban, Kirkpatrick, & Peck, 2001; Dede, 1998; Becker, 
1994), and relegated primarily towards running self- 
contained packaged software programs, or for Internet 
searches (Barron, et al, 2003). Specifically, U.S. schools 
have approximately one computer for every four students 
(Fox, 2005), yet increased access has not raised overall 
student achievement (US DOE, 2004), 

The results appear to indicate that technology use does 
not increase student learning; however, much of the 
research in the area of classroom technology integration 
has focused solely on student outcomes, especially via 
mandated test results (Lowther, et al, 2004; Newman, 
2002; Becker & Riel, 1999). When studies have focused 
instead on teacher preparation and overall 


effectiveness, additional factors emerge (Baylor & 
Ritchie, 2002; Zhao, etal, 2001; Lin, 2001). 

It has been documented that appropriate training of 
teachers in the use of technological tools is an important 
variable in predicting the success of technology 
integration (Faseyitan & Hirschbuhl, 1994), as are several 
other factors such as age, individual readiness, 
motivation, time commitment, and initial technological 
competency (Lee, 2001; Newton, 2003; Rossiter & 
Watters, 2000). One of the challenges in using technology 
effectively in instructional environments is the 
dependency on the technological proficiency of both 
the instructor and students (Leh, 2005). Ample 
documentation exists of failed educational initiatives with 
technological components (Dexter & Riedel, 2003), with 
causes ranging from inadequate technical training to 
unclear or misinterpreted pedagogical goals (Willis, 
1997). The degree of the teacher's technical proficiency 
tends to be a key factor in the eventual outcome of the 
integration of technology in the classroom (Trinkle, 2005) 
Other dominant issues for faculty are the pedagogical 
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benefits derived by the integration of technology in the 
classroom (Surry & Land, 2000). Failure to consider 
pedagogical issues with technology in teaching can 
cause a deviation in the actual learning objectives (Kerr, 
1996). In order to take full advantage of what the use of 
technology offers to the teaching and learning process, 
faculty should align the technology use with the goals of 
teaching (deCastell, Bryson, & Jason, 2002). Developing 
such a relationship requires careful consideration of the 
following factors: alignment between course goals and 
course activities, between course activities and course 
evaluation, between course evaluation and course 
goals, and, importantly, between engaged teaching and 
engaged learning (Sun, 2004). 

Time commitment is often cited as a crucial factor for 
teachers when faced with the prospect of developing the 
skills to appropriately integrate technology in lessons and 
lesson planning (Lawson & Comber, 1999). Teachers 
typically indicate that they are pressed for time in 
preparation, teaching, grading, completing paperwork, 
etc. thus leaving little time to dedicate to using 
technology (Williams, 2000). However, it is also noted that 
the initial requirement for additional time can become an 
investment for time saved in the future, when the 
technology-enhanced courses actually begin to save 
time for teaching (Dexter, Anderson & Becker, 1999). 

The technological competency required for faculty refers 
to the basic skills that would enable the faculty to perform 
the tasks for effectively using technology in the classroom 
(Morrison & Lowther, 2005). An appropriate evaluation of 
faculty technical skills is extremely importantto determine 
readiness and in identifying the skills required to 
accomplish specific tasks; this assessment can assist in 
developing appropriate technology training curriculum 
for faculty (Showers & Joyce, 1996). This training are of 
crucial importance because such requirements are 
often the primary obstacle in a faculty's decision to 
integrate technology in a course or program (Goldberg, 
2000). Furthermore, studies have indicated that teacher 
education programs (TEP) often fail to adequately 
prepare new teachers to teach using technology, which is 
largely due to TEP faculty use of traditional, non¬ 


technology supported instructional practices in required 
courses (Strudler, McKinney & Jones, 1999). 

A variable that is less often considered is that of the time 
spent in or different levels of immersion in a technology 
integration training program (Stuhimann & Taylor, 1999). In 
this context, this factor must be considered to be separate 
and distinct from the variable of overall experience as a 
teacher in the classroom, and/or teacher age. Because 
teacher turnover rates are approaching an all-time high 
with corresponding costs in the $7.3 billion range 
nationally (NCTAF, 2007), there is an appropriate 
responsive level of concern for school districts considering 
implementing long-range teacher development 
projects, and especially for those in the constantly 
changing area of technology. 

Background 

This paper is based on a selected portion of the results of a 
five-year evaluative study of the teachers implementing a 
technology integration program in a rural school district 
located in the Southwestern United States. The overall 
purpose of the evaluation was twofold: (a) to provide 
formative evaluation data to the participant schools to 
serve as a basis for improvement planning and as 
documentation of their accomplishments to 
demonstrate progress; and (b) to provide cumulative 
evidence of the implementation progress and outcomes 
of the participant schools as well as identification of 
exemplary programs. 

The context for the evaluation consisted of eight schools 
receiving specialized training developed by a regional 
consortium and integration training provided by the 
school district's instructional technology department. 
While the consortium training dealt directly with 
developing skills in using technology in the classroom, the 
integration training expanded on this and emphasizes 
using technology to support student-centered teaching 
methods and strategies that promote higher-level 
learning outcomes. 

The final evaluation data were collected from eight 
schools over the academic years 2001 - 2005, and were 
compared to the 2001-2002 (first year) baseline 
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conditions to gauge program implementation progress. 
For the purpose of this study, "novice" as compared to 
"experienced" teachers, is determined by the years of 
exposure, and immersion in the consortium technology 
program. However, the data used for the purposes of 
discussion in this paper was derived from the final year of 
the study (2005-2006) and distinguished the subjects as: 

• "Novice" teachers: those who received the 
consortium training 1 -2 years before the final 
evaluation and summary of the program's 
effectiveness: and 

• "Experienced" teachers: those who received the 
consortium training 4-5 years prior to the final 
evaluation and summary of the program's 
effectiveness, 

Purpose of the Study 

The consortium technology training program's overall 
evaluation was structured around five primary research 
questions that focused on classroom practices, degree 
and type of technology use, academically focused time, 
student engagement, student achievement, and school 
climate. The teacher ability , use of attitudes toward 
technology are also of interest. However, again, for the 
purpose of analyzing the portion of the data for this study, 
the research question investigated the following things: 
What differences exist between novice (to the program 
training) and experienced (in the program training) 
teachers in how and to what degree technology is 
integrated into classroom instruction? 

Methodology 

The evaluation was conducted during the spring 
semester of the 2005-2006 academic year. The overall 
evaluation design for Year 5 reflected those of the Year 1 
baseline study to enable comparative analyses between 
the years. The Year 5 design also included analysis of 
observation data by year(s) of teacher participation in the 
program: Novice 1 to 2 years of experience; Experienced 
4 to 5 years of experience. Both quantitative and 
qualitative data were collected at each of the eight 
schools by trained observers (e.g., retired teachers, district 
staff). The observers spent the majority of their time visiting 


classrooms. A description of the instruments used and 
data collection procedures is provided below. 
Instrumentation and Procedures 
Three instruments in the form of classroom observation 
measures were used to collect the evaluation data for the 
current analysis. These instruments have been fully 
developed and validated. They are as follows: 

Classroom Observation Measures 
Trained observers conducted classroom visits to collect 
frequency data regarding observed instructional 
practices. The visits were targeted or scheduled in 
advance with teachers randomly selected from those 
who participated in the technology training program. 
Selected teachers were instructed to deliver a lesson that 
integrates the use of technology. The data collection 
instruments were the School Observation Measure® 
(SOM), the Survey of Computer Use® (SOU), and the Rubric 
for Student-Centered Activities® (RSCA). The SOM was 
used to collect data regarding overall classroom 
activities, the SOU was used to collect data regarding 
student use of computers, and the RSCA was used to 
capture more detailed information about student- 
centered activities observed during the targeted visits. 
The use of classroom observation instruments are 
described below. 

SOM: The SOM was developed to determine the extent to 
which different common and alternative teaching 
practices are used throughout an entire school or during a 
targeted lesson (Ross, Smith, & Alberg, 1999). During this 
evaluation, it was used to record observations of 
classroom instruction during prearranged one-hour 
sessions in which randomly selected teachers 
demonstrated a prepared lesson for which they were 
asked to use technology. The observers recorded 
classroom events and activities descriptively, not 
judgmentally. Notes forms were completed every 15 
minutes of the lesson to record the use or non-use of 24 
target strategies and the degree to which a high level of 
academically focused class time and a high level of 
student attention/interest was observed, At the 
conclusion of the one-hour lesson, which typically lasted 
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for 45 to 60 minutes, the observer used a SOM Data 
Summary Form to summarize the frequency with which 
each of the strategies was observed. The frequency was 
recorded via a 5-point rubric that ranges from (0) Not 
Observed to (4) Extensively. 

To ensure the reliability of data, observers received a 
manual that provided definitions of terms, examples and 
explanations of the target strategies, and a description of 
procedures for completing the instrument, The target 
strategies include traditional practices (e.g., direct 
instruction and independent seatwork) and alternative, 
predominately student-centered methods associated 
with educational reforms (e.g., cooperative learning, 
project-based learning, inquiry, discussion, using 
technology as a learning tool). The strategies were 
identified through surveys and discussions involving policy 
makers, researchers, administrators, and teachers as 
those most useful in providing indicators of schools' 
instructional philosophies and implementations of 
commonly used reform designs (Ross, Smith, Alberg, & 
Lowther, 2001). 

After receiving the manual and instruction in a group 
session, each observer participated in sufficient 
classroom-based practice, exercises to ensure that 
his/her data are comparable with those of experienced 
observers, in a reliability study (Lewis, Ross, & Alberg, 
1999), pairs of trained observers selected the identical 
overall response on the five-category rubric on 67% of the 
items and were within one category on 95% of the items. 
Further results establishing the reliability and validity of the 
SOM instrument are provided in the Lewis et at. (1999) 
report. 

SCU : A companion instrument to SOM is the Survey of 
Computer Use (SCU) (Lowther & Ross, 2001). The SCU was 
completed as part of the SOM observation sessions 
during which SCU data were also recorded in 15-minute 
intervals and then summarized on an overall data form. 
The SCU was designed to capture exclusively observable 
student access to, ability with, and use of computers 
rather than teacher use of technology. Therefore, in its first 
section, four primary types of data are recorded: (a) 


computer capacity and currency, (b) configuration, (c) 
student computer ability and (d) student activities while 
using computers. Computer capacity and currency is 
defined as the age and type of computers available for 
student use and whether or not Internet access is 
available. Configuration refers to the number of students 
working at each computer (e.g., alone, in pairs, in small 
groups). Student computer ability is assessed by 
recording the number of students who are computer 
literate (/.e., easily used software features/menus, saved 
or printed documents) and the number of students who 
easily use the keyboard to enter text or numerical 
information. 

The next section of the SCU records the types of student 
computer activities, and the subject areas of those 
activities, The computer activities are divided into four 
groups based on broad categories of software used: 
production tools. Internet/research tools, educational 
software, and testing software. Within each category, 
more specific types of software are identified. In the 
Production Tools category, the software types include: 
word processing, databases, spreadsheets, 
draw/paint/graphics, presentation, authoring, concept 
mapping, and planning. In the Internet/research tools 
category, the software include: Internet browser, CD 
reference materials, and communications (e.g., email, 
listservs, chat rooms). In the Educational Software 
category, the software types include: 
drill/practice/tutorial, problem-solving and process tools. 
The Testing Software types include individualized/tracked 
and generic. With this type of recording system, several 
activities can be noted during the observation of one 
student working on a computer. For example, if a student 
gathered data from the Internet, created a graph from 
the data, then imported the graph into a PowerPoint 
presentation, the observer would record that, three types 
of software tools were been observed: Internet browser, 
spreadsheet, and presentation. The frequency with 
which the computer activities and software were 
observed in use was summarized and recorded using a 
five-point rubric that ranges from (0) Not Observed to (4) 
Extensively observed. The subject area of each computer 
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activity was categorized as: language arts, mathematics, 
science, social studies, other, or none. 

The final section of the SCU is an "Overall Rubric" designed 
to assess the degree to which the activity reflects 
"meaningful use" of computers as a tool to enhance 
learning. The definition of meaningfulness is derived from 
the International Society for Technology in Education's 
National Educational Technology Standards for Students 
(NETS-S) (ISTE, 2002). The rubric has four levels: 1) Low- 
level use of computers, 2) Somewhat meaningful, 3) 
Meaningful, and 4) Very meaningful. 

Reliability data for the SCU (Sterbinsky & Burke, 2004), show 
that observer ratings were within one category for 97% of 
the whole-school observations and for 91% of the 
targeted observations. 

RSCA: The Rubric for Student-Centered Activities (Lowther 
&Ross, 2001) was developed for the Center for Research 
in Educational Policy (CREP) as an extension to SOM and 
SCU. The RSCA was used by observers to more closely 
evaluate the strength of application for seven selected 
areas considered fundamental to the goals of increasing 
student-centered learning activities (cooperative 
learning, project-based learning, higher-level 
questioning, experiential/hands-on learning, student 
independent inquiry/research, student discussion, and 
students as producers of knowledge using technology). 
These strategies reflect emphasis on higher-order 
learning and attainment of deep understanding of 
content and whether or not technology was utilized as a 
component of the strategy. Such outcomes are 
intended to be consistent with those likely to be 
engendered by well-designed, real-world linked 
exercises, projects, or problems utilizing technology as a 
learning tool. Each item includes a two-part rating scale. 
The first is a four-point scale, with "1" indicating a very low 
level of application, and "5" representing a high level of 
application. The second is a Yes/No option to the 
question: "Was technology used?" with space provided 
to write a brief description of the technology use. The 
RSCA was completed during the SOM/SCU observation 
periods. 


Data Analysis 

As indicated in the description of SOM, the observation 
procedure primarily focused on 24 instructional strategies 
using a five-point rubric (0 = not observed, 1 = rarely, 2 = 
occasionally, 3 = frequently, and 4 = extensively). In an 
initial analysis, the researchers computed the 
percentage of times a strategy was not observed (rubric 
category = 0) vs. observed (categories 1-4 combined). 
Chi-square analyses were conducted to compare 
Novice vs. Experienced results. To examine the full, five- 
category breakdown of the SOM, t-tests for independent 
samples were performed on each item to determine 
whether significant differences existed between the 
classroom practices of Novice and Experienced 
teachers. In addition, Effect Sizes (ES) using Cohen's d 
formula (Cohen, 1988) were computed to determine the 
educational importance of differences (an ES having an 
absolute value greater than .25 is considered to be 
educationally important). 

Results 

A summary of the results grouped by the 3 observation 
instruments namely School Observation Measure (SOM), 
Survey of Computer Use (SCU), and Rubric for Student- 
Centered Instruction is discussed here. The data for 58 
classroom observations were collected with SOMs, SCUs, 
and RSCAs during prearranged sessions in which teachers 
were asked to implement a lesson using technology, 
Results from each measure are described in the following 
section. The data presented were compared on the 
basis of teacher experience with the technology training 
program: novice (n = 28) vs. Experienced (n = 30). 

School Observation Measure [SOM] 

The greatest difference when examining the data by 
experience level, was that the novice teachers used 
higher-level questioning strategies more often than the 
experienced teachers: 75% observed in the novice 
teachers as compare to nearly 57% not observed in the 
experienced teachers (Table 1). However there were 
other differences worth noting; in particular the use of 
experiential, hands-on learning strategies by the novice 
teachers exceeded that of the experienced teachers by 


31 


i-manager’s Journal of Educational Technology, Vol. 4 • No. 3 • October - December 2007 






RESEARCH PAPERS 



Not Observed 

Observed 

Instructional Strategies 

Novice 

Experienced 

Ncvice 

Experienced 

Direct instruction 

10.7 

26.7 

89.3 

73.3 

Teacher as 
coach/facilitator * 

10.7 

30.0 

89.2 

70.0 

Use of higher level 
questioning *** 

21.4 

60.0 

78.6 

39.9 

Technology as a 
learning tool ** 

25.0 

46.7 

75.1 

53,3 

Higher level 
instructional 

feedback *** 

25.0 

56.7 

75.0 

43.3 

Computer for 
instructional delivery 

25.0 

30.0 

74.9 

70.0 

Experiential, hands on 
learning ** 

42.9 

70.0 

57.1 

30,0 

Ability groups 

50.0 

70.0 

49.9 

29.9 

Student discussion 

57,1 

60.0 

42.9 

39.9 

Cooperative/ 
Collaborative learning 

60.7 

66.7 

39.4 

33.4 

Integration of subject 
areas 

64.3 

66.7 

35.7 

33.3 

Independent seatwork 

67.9 

43,3 

32.1 

56.6 

Work centers 

67.9 

73.3 

28,6 

26,6 

Project -based learning 

71.4 

70.0 

28.6 

30.1 

Individual tutoring 

75.0 

76.7 

25.1 

23.4 

Independent 

inquiry/research 

75,0 

66,7 

25.1 

33.3 

Team teaching 

78.6 

83.3 

21.4 

16.7 

Systematic, individual 
instruction 

78,6 

86,7 

21,4 

13.2 

Multi -age grouping 

82,1 

73.3 

17.8 

26,7 

Parent/community 

involvement 

89.3 

96.7 

10.7 

3.3 

Performance 

assessment 

89.3 

76.7 

10.7 

23.3 

Student self - 

assessment 

92.9 

90.0 

7.2 

9.9 

Sustained 

writing/composition 

96.4 

93.3 

3.6 

6.6 

Sustained reading 

96.4 

96.7 

3.6 

3.3 


*p<.05„ **p<.01, ***p<.001 
Note . Sorted from highest to lowest proportion of Novice Teacher Observed 

Table 1 .School Observation Measure (SOM) Proportion of 
not observed (0) vs. observed (1-4) Strategies 

over 27%. Experiential, hands-on learning was not 
observed in seventy percent of the experienced 
teachers' classrooms. Other areas where the novice 
teachers exceeded use of specific learning strategies 
were in the use of technology tools (over 75% observed in 
novice teachers versus almost 47% not observed in 
experienced teachers), and the teacher in the role of 
coach/facilitator: over 89% observed in novice teachers 


Computer configuration Novice Teachers Experienced Teachers 

Classrooms most frequently had the following number 


of computers or digital tools 



None 

0.0 

3.3 

One 

10.7 

20.0 

2-4 

35.7 

20.0 

5-10 

32.1 

33.3 

11 or more 

21.4 

23,3 

Classroom computers were most 

frequently 


Up-to-date* 

96.4 

86.7 

Aging, but adequate 

3.6 

10.0 

Outdated/ limited 

0.0 

0.0 

capacity 



No computers were 

0.0 

3.3 

observed 



Classroom computers were most 

frequently 


Connected to the Internet* 

100.0 

83.3 

Not connected to the 

0.0 

13.3 

Internet 



No computers were 

0.0 

3.3 

observed 



Student computer use 

Classroom computers or digital tools were most frequently used by: 

Few (less than 10%) 

10.7 

23.3 

students 



Some (about 10-50%) 

21.4 

13.3 

students 



Most (about 51-90%) 

10.7 

10.0 

students 



Nearly all (91-100%) 

46.4 

33.3 

students ** 



Students need not use 

10.7 

20.0 

computers 



Students most frequently worked with computers or digital tools: 

Alone 

57.1 

53.3 

In pairs 

17.9 

10,0 

In small groups 

7.1 

3.3 

Students did not use 

17.9 

33.3 

computers 



Student computer literacy skills were most frequently: 


Poor 

3.6 

0.0 

Moderate 

25.0 

30.0 

Very good** 

50.0 

36.7 

Not observed 

21.4 

33.3 

Student computer literacy skills were most frequently: 


Poor 

3.6 

3,3 

Moderate 

7,1 

23.3 

Very good*** 

46.4 

16.7 

Not observed 

42.9 

56.7 


Novice Teacher N = 28; Experienced Teacher N = 30 
*P<0.05,**p<0.01, ***p<0.001 


Table 2. Survey of Computer Use (SCU) Data Summary: 
Computer Configuration 

compared to 30% not observed in the experienced 
teachers. 

In all other areas that were measured by the SOM, few 
differences were observed. 

Survey of Computer Use [SCU] 

There were some differences between novice and 
experienced teachers in a few areas measured by the 
SCU (Tables 2-6). Table 2 shows that there were some 
differences in availability and use of computer hardware 
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Computers/Digital Tools 
Used by Students 

Experience 

Level 

Not 

Observed 

Observed 


Novice 

46.4 

53.6 

Desktop Computers 

Experienced 

50.0 

50 


Novice 

60.7 

39.2 

Laptop Computers 

Experienced 

56.7 

43.3 

Personal Data Assistants 

Novice 

96.4 

3.6 

(PDA) 

Experienced 

100.0 

0 


Novice 

100.0 

0 

Graphing Calculator 

Experienced 

100.0 

0 

Information Processor 

Novice 

100.0 

0 

(e.g., Alphaboard) 

Experienced 

100.0 

0 

Digital Accessories ( e.g.. 

Novice 

85.7 

14.3 

camera, scanner, probes) 

Experienced 

93.3 



Table 3. Survey of Computer Use (SCU) Data Summary: 
Computer/Digital Tools Used by Students 


and Internet access, and that novice teachers had a 
greater percentage of up-to-date computers and 100% 
access to the Internet in their classrooms. Also noteworthy, 
more students (91-100%) in the novice teachers' 
classrooms demonstrated more frequent use of 
computers or other digital tools (46.4% of the time), as well 
as demonstrated computer literacy skills at a Very Good 
level of 50% of the time. Students in the novice teachers' 
classrooms also demonstrated better keyboard using 
skills at a higher level (46.4% compared to 16.7% in the 
experienced teachers' classrooms). 

While other areas measured by the SCU (Tables 3 - 5) did 
not indicate significant differences as Observed versus 
Not Observed, the year five data showed that teachers, 
novice to the program integrated the students' use of 
computers into their instruction in Meaningful ways (25% 
of teachers novice to the program compared to just over 
13% of teachers experienced in the program), and at in 
Very Meaningful ways (nearly 18% of novice teachers to 
just over 3% of experienced teachers) at a difference in 
ratio (Table 6). 

Rubric for Student-Centered Activities (RSCA) 

The RSCA results revealed, again, an increased usage of 
technology to support higher-level questioning by the 
teachers who were novice to the program (nearly 43%) 
when compared to the teachers who had 4-5 years of 


Student computer Activities 

Experience 

Not 

Observed 

by software used 

level 

observed 


production Tools 




Word processing 

Novice 

89.3 

10.7 


Experienced 

83.3 

16.6 

Database 

Novice 

100 

0 


Experienced 

100 

0 

Spread sheet 

Novice 

100 

0 


Experienced 

100 

0 

Draw/panit/Graphics 

Novice 

92.9 

7.2 


Experienced 

86.7 

13,3 

Presentation^,g., MS power 

Novice 

92.9 

7.1 

point)** 

Experienced 

90 

9.9 

Authoring (e.g., Hyper 

Novice 

100 

0 

Studio) 

Experienced 

100 

0 

Concept Mapping (e.g.. 

Novice 

100 

0 

Inspiration) 

Experienced 

96.7 

3.3 

Planning (e.g., MS project) 

Novice 

100 

0 


Experienced 

100 

0 

Internet/Research Tools 

Internet Browser (e.g.. 

Novice 

75 

25 

Netscape)** 

Experienced 

73.3 

26.7 

CD Reference 

Novice 

100 

0 

(encyclopedias, etc.) 

Experienced 

100 

0 


Novice 

100 

0 

Communications 

Experienced 

100 

0 

Educational Software 

Drill/ practice/ Tutorial 

Novice 

64.3 

35.7 


Experienced 

73.3 

26.6 

Problem solving (e.g. Slim 

Novice 

100 

7.2 

city) 

Experienced 

100 

13.2 

Process Tools(Geometer's 

Novice 

100 

0 

sketchpad, etc.) 

Experienced 

100 

3.3 

Testing Software 

Individualized/Traked (e.g,. 

Novice 

100 

0 

Accelerated Reader) 

Experienced 

96.7 

3.3 

Generic 

Novice 

100 

0 


Experienced 

96.7 

3.3 


Table 4. Survey of Computer Use (SCU) Data Summary: 
Student Computer Activities by Software Used 


experience in the program (33.3%) (Table 7). Also 
noteworthy is the higher quality with which teachers 
implemented cooperative and project-based learning 
after five years as compared to the first year of program 
participation. These results suggest that the program was 
successfully preparing teachers to implement above 
average student-centered activities. In other areas, there 
were no significant differences between the novice and 
experienced teachers, or, as in the use of Cooperative 
Learning, experienced teachers used this strategy at a 
higher rate (40%) compared to the novice teachers 
(25%). 

Analysis and Evaluation 

The analysis of the data is derived directly in association 
with the research question: What differences exist 


33 


i-manager’s Journal of Educational Technology, Vol. 4 • No. 3 • October - December 2007 




































RESEARCH PAPERS 


Subject 
Areas of 
computer 
Activities 


Experience 

None 

Other 

Language 

Mathematics 

Science 

Social 

Total 

Level 






Studies 

Observed 


Production 

Tools 

Internet/ 

Research 

Tools 

Educational 

Software 

Testing 

Software 


Novice 

Experienced 

Novice 

Experienced 


50 

60 


75 

60 


Novice 60.7 

Experienced 66,7 
96 A~ 

86,7 


Novice 

Experienced 


10.7 

6.7 

10.7 
3.3 

3.6 

3.3 

0 

3.3 


25 

13.3 

7.1 

13.3 

25 

20 

0 

3.3 


10.7 
23.3 

3.6 

16.7 

17.9 

20 

3.6 

0 


10.7 

6.7 

3.6 

6.7 

7.1 

6.7 
0 

3.3 


10.7 

6.7 

7.1 

6.7 

10.7 
3,3 

0 

3,3 


67.8 

53.3 

32.1 
46.7 

64.3 

53.3 
3.6 

13.2 


*Note: Item percentages may not total 100% because of 
missing input from some respondents 

Table 5. Survey of Computer Use (SCU) Data Summary: Subject Areas of Computer Activities 


Overall meaningful 
use of computers 

Experience 

Level 

Not 

observed 

Rdrely 

Occasionally 

Frequently 

Extensively 

Total 

Observed 



75.0 

10.7 

0.0 

7.1 

7.1 

24.9 

Low level use of 
computers 

Novice 

Experienced 

83.3 

10.0 

0.0 

3.3 

3,3 

16.6 

Somewhat meaningful 

Novice 

75.0 

7.1 

3.6 

7.1 

7,1 

24.9 

use of computers 

Experienced 

80.0 

0.0 

6.7 

10.0 

10,0 

20 

Meaningful use of 

Novice 

50.0 

3.6 

3.6 

17.9 

17,9 

50.1 

computers* 

Experienced 

56,7 

3,3 

6.7 

16,7 

16.7 


Very meaningful 

Novice 

64,3 

7.1 

3.6 

7.1 

7.1 

35.7 

use of computers 

Experienced 

73.3 

20.0 

3.3 

20.0 

20.0 

26.6 


*p<.05, **p<.01 


Table 6. Survey of Computer Use (SCU) Data Summary: Overall Meaningfulness of Computer Activities 


Items 

Experience 

Level 

% 

observed 

Rubric Rating **- 

percentage observed 

Percentage that 
used Technology 


1 

2 

3 

4 


Novice 

39,3 

7.1 

0.0 

14.3 

17,9 

25.0 

Co-operative Leorning 

Experienced 

56.6 

3.3 

23.3 

20.0 

10.0 

40.0 

Project-based Learning 

Novice 

Experienced 

28.7 

40.1 

3.6 

6.7 

3.6 

6.7 

17.9 

20.0 

3.6 

6.7 

28.6 

33.3 

High- level Questioning 

Novice 

82.2 

7.1 

17.9 

28.6 

28.6 

42.9 

Strategies* 

Experienced 

60.0 

13.3 

16.7 

20.0 

10.0 

33.3 

Experiential, hands-on 

Novice 

53.6 

0.0 

10.7 

14.3 

28.6 

25.0 

Learning 

Experienced 

33.0 

0.0 

10.0 

10.0 

13.3 

30.0 

Independent Inquiry/ 

Novice 

32.1 

3.6 

7.1 

7.1 

14.3 

21.4 

Resedrch 

Experienced 

33.3 

6.7 

13.3 

3.3 

10.0 

30.0 

Student Discussion 

Novice 

Experienced 

42.9 

39.9 

7.1 

13.3 

14.3 

13.3 

17.9 

10.0 

3.6 

3.3 

17.9 

23.3 

Students as producers of 

Novice 

32.2 

3.6 

0.0 

14.3 

14.3 

*** 

knowledge 

Experienced 

60.0 

20.0 

3.3 

26.7 

10.0 

■kick 


Novice Teachers (2005-2006) N = 28 Experienced Teachers (2005-2006) N = 30 *p < 

**Rating scale: 1 = limited application: 2 = somewhat limited application: 3 = somewhat strong application: 4 = strong application. 

*** Students as Producers of Knowledge” is not included because use of technology was not a required component. 


Table 7. Rubric for Student Centered Activities (RSCA) Item Ratings by Percentage Observed 


between novice (to the program training) and 
experienced (in the program training) teachers in how- 
and to what degree technology is integrated into 
classroom instruction? 


When examining the results by level of experience in the 
consortium technology training program, one notable 
difference was revealed during the classroom 
observations: the novice teachers in the program (1-2 
years) more frequently used higher-level questioning 
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strategies than the teachers with 4-5 years of experience 
in the program. 

Other differences worth noting were in the use of 
experiential, hands-on learning strategies by the novice 
teachers, exceeding that of the experienced teachers, 
where this instructional strategy was not observed in a 
majority of the experienced teachers 1 classrooms. Novice 
teachers also used technology tools at an increased rate 
than that of teachers experienced in the program and in 
the teacher in the role of coach/facilitator. 

Some differences between novice and experienced 
teachers also were shown in the availability and use of 
computer hardware and Internet access, and with novice 
teachers' students more frequently using computers or 
other digital tools, increased computer literacy skills, and 
higher levels of keyboarding skills. 

In another area measured, the data showed that 
teachers novice to the program integrated the students' 
use of computers into their instruction in meaningful and 
in very meaningful ways at a higher ratio. 

Conclusions 

The findings that, in some areas measured, the teachers 
with less exposure, or immersion, in the technology 
training program outperformed the teachers with longer, 
prior experience in the program poses interesting 
problems, and raises questions in regards to what 
additional variables contributed to the outcomes that 
were either totally unrelated to the program training or 
were peripheral factors that affected the results as 
measured by the instruments used. One speculative 
cause could be that the performance by the teachers 
who were novice to the program reflects refinement of 
the professional development that the novice teachers 
received; however, this variable was not measured, and it 
cannot be readily assumed that the training program 
changed significantly during the five year 
implementation. 

Other variables, such as the age or experience of the 
teachers identified as novice in this study and introduced 
later into the program, were not considered when 
demographic data were gathered. The possibility that the 


novice teachers received increased mentoring by 
experienced teachers either outside or within their school 
environment is also a variable worthy of consideration, but 
again was not measured within the confines of this study. 
Suggestions for Further Research 
Because the five year program as a whole was deemed a 
success in the district where it was implemented, the 
results of this focused meta-study cannot be used to call 
into question the overall effectiveness of the technology 
training program itself. However, the questions raised by 
analysis of the better performance of the "newcomers" 
relative to the "old-timers" to the program in some areas 
should be investigated. A more thorough demographic 
portrait and comparison between the two identified 
groups would be a very good start in understanding why 
the relative level of immersion in the technology training 
program was not a deciding factor in many important 
areas that were specifically targeted by the training 
program, Also, any changes made in the training 
program should be thoroughly investigated and 
identified to determine if critical factors were contributed 
due to such changes. 

References 

[1] . Barron, A. E., Kemker, K., Harmes, C., &Kalaydjian, K. 
(2003). Large-scale research study on technology in 
K-12 schools: Technology integration as it relates to the 
National Technology Standards. Journal of Research on 
Technology in Education, 35(4), 489-507. 

[2] . Baylor, A. and D. Ritchie (2002). "What factors 
facilitate teacher skill, teacher morale, and perceived 
student learning in technology-using classrooms?" 
Computers & Education 39(4): 395-414. 

[3] , Becker, H. J. (1994). How exemplary computer-using 
teachers differ from other teachers: Implications for 
realizing the potential of computers in schools. Journal of 
Research on Computing in Education, 26 ,291 - 321 . 

[4] , Becker, H. J., & Riel, M. M. (1999). Teacher 
professionalism, schoolwork culture and the emergence 
of constructivist-compatible pedagogies [PDF file]. 
Center for Research on Information Technology and 
Organizations. Available online at 


35 


i-manager’s Journal of Educational Technology, Vol. 4 • No. 3 • October - December 2007 






RESEARCH PAPERS 


http://www.crito.uci.edu/tlc . 

[5] , Cuban, L., Kirkpatrick, H., & Peck, C. (2001). High 
access and low use of technologies in high 
school classrooms: Explaining an apparent paradox. 
American Educational Research Journal, 38, 813-834. 

[6] , de Castell, S., Bryson, M., & Jason, J. (2002). Object 
lessons: Towards an educational theory of technology. 
First Monday, 7(1). Retrieved Oct. 24, 2006 from 
http://firstmonday.org/issues/issue7_l/castell/index.html. 

[7] , Dede, C. (Ed.) (1998). Learning with technology: The 
1998 ASCD Yearbook. Alexandria, VA: Association for 
Supervision and Curriculum Development. 

[8] , Dexter, S. & Riedel, E. (2003). Why improving pre¬ 
service teacher educational technology preparation 
must go beyond the college's walls. Journal of Teacher 
Education, 54(4), p.p. 334-346. 

[9] , Dexter, S., Anderson, R. E., & Becker, H. J. (1999). 

Teachers' views of computers as catalysts for Changes in 
their teaching practice, Journal of Research on 
Computing in Education, 31 [ 3), 221-239. 

[10] . Faseyitan, J.J. & Hirschbuhl, S.O. (1994). Faculty 
uses of computers: Fears, facts and perceptions. THE 
Journal 21(9), 64-65, 

[11] , Fox, E. (May, 2005). Tracking U.S. Trends. Education 
Week's Technology Counts 2005. Marion, OH: Editorial 
Projects in Education. 

[12] , Goldberg, M.L. (2000). Educational technology and 
distance education at UW Bothelklnitial findings, 
observations, and recommendations. Retrieved Oct. 30, 
2006fromhttp://faculty. uwb.edu/mgoldberg/edtech/rep 
ort.html. 

[13] . International Society for Technology in Education 
(ISTE) (2003). National Educational Technology Standards 
for Students (NETS*S): Technology Foundation Standards 
for all Students. Available online at 
http://cnets.iste.org/students/s_stands.html 

[14] , Kerr, S. T. (1996). Toward a sociology of educational 
technology. In Jonassen, D.H. (Ed.), Handbook of 
research for educational communications and 
technology, (pp. 143-169), NewYork: Simon 8cSchuster. 


[15] . Lawson, T. & Comber, C. (1999). Superhighways 
Technology: personnel factors leading to successful 
integration of information and communications 
technology in schools and colleges. Technology, 
Pedagogy and Education, 8(1), 41 -53. 

[16] . Lee, K.T. (2001 ) Information technology integration in 
teacher education: Supporting the paradigm shift in 
Hong Kong. Asia-Pacific Journal of Teacher Education & 
Development, 4(1), 157-78. 

[1 7]. Leh, A. (2005) Lessons learned from service learning 
and reverse mentoring in faculty development: A case 
study in technology training. Journal of Technology and 
Teacher Education, 13(1), 57-63. 

[18] , Lewis, E. M., Ross, S. M., & Alberg, M. (July, 1999). 
School Observation Measure: Reliability analysis. 
Memphis, TN: Center for Research in Educational Policy, 
University of Memphis. 

[19] . Lin, X. (2001). "Reflective adaptation of a technology 
artifact: A case study of classroom change." 
Cognition and instruction 19(4): 395-440, 

[20] . Lowther, D. L., & Ross, S. M. (2001). Survey of 
Computer Use (SCU). Memphis, TN: Center for Research 
in Educational Policy, University of Memphis. 

[21 ]. Lowther, D. L. & Ross, S. M. (2001). Rubric for Student- 
Centered Learning (RSCA ]. Memphis, TN: Center for 
Research in Educational Policy, University of Memphis. 

[22] , Lowther, D. L., Ross, S. M., Walter, J. W., McDonald, A. 
J., &. Wang, L. W. (2002). Tennessee Technology Literacy 
Challenge Fund: Evaluation Report. Memphis, TN: 
Center for Research in Educational Policy, University of 
Memphis, 

[23] , Morrison, G.R. & Lowther, D. L., (2005). Integrating 
Computer Technology into the Classroom (3 rd Ed.), 
Englewood Cliffs, NJ: Merrill/Prentice Hall. 

[24] . Newman, H. (2002, February 26). Computers used 

more to learn than teach. Detroit Free Press. Available 
online at 

http://www.freepress.com/news/education/newman26_ 
20020226.htm 

[25] , Newton, R. (2003). Staff attitudes to the 


i-manager’s Journal of Educational Technology, Vol. 4 • No. 3 • October - December 2007 


36 







RESEARCH PAPERS 


development and delivery of e-learning, New Library 
World, 104(10), 412-425, 

[26] . Norris, C., Sullivan, T. , Poirot, J., Soloway, E. (2003) 

No Access, No Use, No Impact: Snapshot Surveys of 
Educational Technology in K-l 2. Journal of Research on 
Technology in Education, 36(1), 15-28, 

[27] , Ross, S. M., Smith, L. J„ & Alberg, M. (1999). The 
School Observation Measure (SOM' 6 ]. Memphis, TN: 
Center for Research in Educational Policy, University of 
Memphis. 

[28] , Ross, S. M., Smith, L., Alberg, M., & Lowther, D. (2004) 

Using Classroom Observations as a Research and 
Formative Evaluation Tool in Educational Reform: The 
School Observation Measure. In S. Hilberg and H. 
Waxman (Eds,) New Directions for Observational 
Research in Culturally and Linguistically Diverse 
Classrooms (pp. 144-173). Santa Cruz, CA: Center for 
Research on Education, Diversity 8c Excellence. 

[29] , Rossiter, D. 8c Watters, J.J. (2000). Technological 
literacy: foundations for the 21 st century. Brisbane: 
Queensland University of Technology. 

[30] . Showers, B., 8c Joyce, B. (1996). The evolution of 
peer coaching. Educational Leadership, 54 (6), 12-16. 

[31] . Sterbinsky, A., 8c Ross, S. M., (2003). Summary of 
CSRTQ Reliability Studies. Technical Report. Memphis, TN: 
Center for Research in Educational Policy, University of 
Memphis, 

[32] , Strudler, N. B., Mckinney, M. O. 8c Jones, W.P (1999) 

First-year teachers' use of technology: 

Preparation, expectations and realities. Journal of 
Technology andTeacherEducation (1999) 7(2), 115-1 29. 

[33] , Stuhlmann, J. M., 8c Taylor, H. G. (1999). Preparing 
technically competent student teachers: A three-year 


study of interventions and experiences, Journal of 
Technology and Teacher Education, 7(4), 333-350. 

[34] , Sun, J.R. (2004). Turning a regular (face-to-face) 
course into a more engaging blended (hybrid) course. 
Paper presented at the Ohio Commons for Digital 
Education 2004 - The Convergence of Libraries, Learning 
and Technology Conference March 8-9,2004, 

[35] . Surry, D.W. 8c Land, S.M. (2000). Strategies for 
motivating higher education faculty to use technology. 
Innovations in Education and Teaching International, 
37(2), 145-153. 

[36] . Trinkle, D.A. (2005). The 361° Model for transforming 
teaching and learning with technology. Educause 
Quarterly, 28(4), 1 8-25. 

[37] , U.S. Department of Education (DOE), (November, 

2004) U.S. Department of Education FY 2004 
Performance and Accountability Report, Washington 
D.C. Available online 

athttp://www.ed.gov/about/reports/annual/2004report/in 
dex.html 

[38] , Williams, C. (2000). Internet access in U.S. public 
schools and classrooms: 1994-1999. (NCES 2000-086). 
U.S. Department of Education, Washington, DC: National 
Center for Education Statistics. 

[39] , Willis, E. M. (1997). Technology: Integrated into, not 
added onto, the curriculum experiences in pre-service 
teacher education. Computers in the Schools, 13(1-2), 
141-53. 

[40] . Zhao, Y., Byers, J., Mishra, R, Topper, A., Chen, H., 
Enfield, M., etal. (2001 .Winter). What do they know? A 
comprehensive portrait of exemplary technology-using 
teachers, Journal of Computing in Teacher 
Education, pp. 2537. 


37 


i-manager’s Journal of Educational Technology, Vol. 4 • No. 3 • October - December 2007 






RESEARCH PAPERS 


ABOUT THE AUTHORS 

* Assistant Professor, College of Education, University of Memphis, Memphis. 

** Associate Professor, College of Education, University of Memphis, Memphis. 

* * * Project Coordinator, Center for Research in Educational Policy, University of Memphis, Memphis. 

Dr. Lee Allen is an Assistant Professor of Instructional Design and Technology, and has served as the Special Assistant to 
the Chair of the Instruction and Curriculum Leadership Department for Online Learning and Technology Integration. 
Dr. Allen has also served as a school district assistant superintendent for technology in Dallas, TX, and as a technology 
administrator in Santa Fe, NM. 

Dr. Deborah Lowther is an Associate Professor of Instructional Design and Technology and Director of Graduate Studies 
for the Instruction and Curriculum Leadership department at the University of Memphis' College of Education. Dr. 
Lowther is also the Senior Faculty Researcher at the Center for Research in Educational Policy [CREP] at the University 
of Memphis. 

Dan Strahl is Project Coordinator at the Center for Research in Educational Policy (CREPj at the University of Memphis. 
He is responsible for numerous research and evaluation projects, primarily with focus in the area of technology 
integration in schools. 



i-monager’s Journal of Educational Technology, Vol. 4 • No. 3 • October - December 2007 


38 









